The Effect of Gain Variation in Improving Learning Speed of Back Propagation Neural Network Algorithm on Classification Problems

We proposed a method for improving the performance of the back propagation algorithm by introducing the adaptive gain of the activation function. In a ‘feed forward’ algorithm, the slope of the activation function is directly influenced by a parameter referred to as ‘gain’. In this paper, the influence of the adaptive gain on the learning ability of a neural network is analysed. Multi layer feed forward neural networks have been assessed. Physical interpretation of the relationship between the gain value and the learning rate and weight values is given. Instead of a constant ‘gain’ value, we proposed an algorithm to change the gain value adaptively for each node. The efficiency of the proposed method is verified by means of simulation on three classification problems. The results show that the proposed method significantly improves the learning speed of the general back-propagation algorithm.